REMARKS 

Claims 1-5, 11-15 and 19-23 are pending in this application, of which 
Claims 1,11 and 19 are the independent claims. Claims 6-10 and 16-18 have been 
canceled, without prejudice or disclaimer of subject matter, and will not be mentioned 
further. Claims 1-5 and 11-15 have been amended to define still more clearly what 
Applicants regard as their invention. Claims 19-23 have been added to assure Applicants a 
fuller measure of protection of the scope to which they deem themselves entitled. A 
substitute specification is submitted herev^ath in both a clean and a marked version; no new 
matter has been added. 

Claim 1-3, 5 and 11-13 were rejected under 35 U.S.C.§ 102(b) as being 
anticipated by U.S. Patent 5,416,535 (Sato et aL), Claims 4 and 14 were rejected under 35 
U.S.C. § 103(a) as being unpatentable over Sato in view of U.S. Patent 5,500,794 (Fujita et 
al.), and Claim 15 was rejected under Section 103(a) as being unpatentable over Sato in 
view of U.S. Patent 6,400,280 (Osakabe). 

As is described in the present application in greater detail, the present 
invention is directed to the solution of a problem that has appeared with progress toward 
the presence of large, complex networks in the home. The connection of highly variegated 
appliances to such a network, and the desire to be able to control those appliances using the 
network, leads to a problem in that such appliances, being of different types and often from 
difference manufacturers, firequently use incompatible systems for control, incompatible 
standards, and the like. A control device that provides all the different fimctions necessary 
to be able to control such an array of disparate appliances is it self firequently cumbersome 
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and inconvenient to use, for example requiring the user to actuate a button many times to 
reach the function that is needed in a particular instance. 

In the aspects of the present invention to v^hich Claims 1 and 1 1 are 
respectively directed, a communication system uses a control apparatus (e.g., 120) that 
receives a control signal from an operation apparatus (e.g., 130), i.e., an apparatus to be 
controlled, where the latter has a first operation unit /or operating its operation panel and a 
second operation unit for operating a specific function of the controlled apparatus (e.g., 100 
and 1 10). According to both Claims 1 and 1 1, the control apparatus determines, according 
to the control signal, v^hether or not to transmit a command for operating the specific 
function from the control apparatus to the controlled apparatus (e.g., page 19, line 21 to 
page 22, line 9).* 

For example, independent Claim 1 is directed to a communication system, 
that comprises a control apparatus adapted to receive an operation panel for operating a 
controlled apparatus from the controlled apparatus, and to display the operation panel on a 
display unit. Also provided is an operation apparatus having a first operation unit for 
operating the operation panel and a second operation unit for operating a specific function 
of the controlled apparatus. According to Claim 1, the control apparatus is adapted to 
receive a control signal from the operation apparatus, and to determine, according to the 
control signal, v^hether or not to transmit a command for operating the specific function 
from the control apparatus to the controlled apparatus. 



1/ It is to be understood that the claim scope is not limited by the details of the particular 
embodiments referred to, v^hich are mentioned solely for the purpose of facilitating 
explanation. 
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At the least, the mentioned feature of Claim 1 is not taught by anything 
believed to be in Sato. This patent relates to a system in which each of a plurality of AV 
apparatuses (61-64 in Fig. 1) is connected to an amplifier 10, so as selectively to provide 
output of each apparatus to a speaker 71 or a monitor 70. This system uses a remote 
controller R for remotely controlling each of the AV apparatuses. The remote controller is 
used to operate a control panel displayed on the monitor (see the Abstract). Hov^ever, the 
control panel is generated not by the AV apparatus but by a graphic controller 25 v^hich is 
included in the amplifier 10 (Fig. 2, col. 7, lines 44-64, and coL 9, lines 27-37). Applicants 
submit that nothing in Sato would teach or suggest the control apparatus recited in Claim 1, 
which receives the control panel of a controlled apparatus fro, the controlled apparatus. 

For at least that reason, Claim 1, and similarly Claim 1 1, are deemed to be 
clearly allowable over Sato. 

Independent Claim 19 is a method claim corresponding to Claim 11, and is 
believed to be allowable for the same reasons as is Claim 1 . 

A review of the other art of record has failed to reveal anything which, in 
Applicants' opinion, would remedy the deficiencies of the art mentioned above, as a 
reference against the independent claims herein. Those claims are therefore believed 
patentable over the art of record. 

The other claims in this application are each dependent from one or another 
of the independent claims discussed above and are therefore believed patentable for the 
same reasons. Since each dependent claim is also deemed to define an additional aspect of 
the invention, however, the individual consideration or reconsideration, as the case may be, 
of the patentability of each on its own merits is respectfully requested. 
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In view of the foregoing amendments and remarks, Applicants respectfully 
request favorable reconsideration and passage to issue of the present application. 

Applicants' undersigned attorney may be reached in our New York office by 
telephone at (212) 218-2100. All correspondence should continue to be directed to our 
below listed address. 

Respectfully submitted, 



Attorney for Applicants 
Registration No. \ 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 10112-3801 
Facsimile: (212)218-2200 



NY MAIN 406454 vl 
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A.N. 09/819 J 
Atty. Docket No. 03500.0^pf 
Substitute Specification 24 Marcn 26\ 

(marked version) 

COMMUNICATION SYSTEM, COMMUNICATION APPARATUS, 

AND COMMUNICATION METHOD .,,^r=^ 

RECEIVED 
iAR 3 1 2004 

BACKGROUND OF THE INVENTION 

T®etinology Center 2600 

Field of the Invention 

The present invention relates to a system, an apparatus, and a method for 
remotely operating a desired electronic device by means of a graphical user 
interface. 

Related Backgroxmd Art 

In recent years there have been developed high-speed and advanced digital 
interfaces for transmitting data expected to have real time properties such as a 
dynamic image (moving image) or a sound or voice and data on which it is 
expected that a user can rely, for to have r eliability on communications such as a 
static image and a control command. A digital interface conforming to the 
lEEEl 394-1 995 standards is one of these digital interfaces. 

Furthermore , there recently there has been suggested that electronic devices 
in a house having this type of digital interfaces (for example, AV (audio-visual) 
equipment, household electric appliances, and a personal computer, etc.) arc might 
be connected with each other so as to construct a home network. 

Wli c n the In such a home networ k p re vails among us , a device functioning 
as a center of the home network [[is]] may, for example, be a television set. If so, 
the television set provides a user with graphical user interfaces for remotely 
operating other devices^ and the user operates a graphical user interfaces displayed 
on the television set to operate a device corresponding to the graphical user 
interface. 
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There_are, however, arc a plurality of types of devices cormectable to the 
home network^ and they have operating environments different from each other. 
Furthermore, even if the devices have the same function, their operating 
environments depend upon their manufacturers or model types. 
5 Therefore, there is such a problem in that the television set is required to 

cope v^th graphical user interfaces depending upon manufacturers or model types 
flexibly. 

In addition, there is also a problem m that the different operation methods 
for each graphical user interface will cause [[an]] operation to be complicated. To 

1 0 solve this problem, there can be a method of operating each graphical user interface 
by using direction keys and selection keys. 

This method, however, has [[a]] tiie problem of requiring the direction keys 
to be operated by a user a plurality of number of times until a desired function is 
selected, [[by]] as a result of which it takes much time for executing to execute the 

1 5 desired function. Particularly if a plurality of operations are required to be 

executed continuously, the operations are complicated, by which and so quick 
operations cannot be performe d, which is disadvantageous[[ly]]. 



SUMMARY OF THE INVENTION 

2 0 An object of the present invention is to solve the above-described problems. 

According to one aspect, the present invention which achieves this object 
relat e d to can be a communication system, having comprising : a controll e d d e vic e 
fo r managing an ope r ation pan e l having a p lurality of display elements 
con ' csponding to prcdctemiincd ope r ations, a controller for accepting an operation 

25 for th e operation panel suppli e d by tlie controlled device, and an o p e r ation 

appai - atus having a fi r st ope r ation unit fo r ope r ating tli e operation panel and a 
second operation unit fo r specifying a specific ope r ation, wherein said controller 
t r ansmits a fi r st control signal indicating that any of said display clem e nt has been 
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operat e d on the basis of aii operation for the first operation uiait to the controlled 
devic e and t r ansmits a second cont r ol signal for spccifi^ing any of the sp e cific 
o p e r ations on the basis of an ope r ation fo r tlic second ope r ation miit to th e 
controlled device a control apparatus adapted to receive an operation panel for 
5 operating a controlled apparatus from the controlled apparatus, and to display the 
operation panel on a display unit: and an operation apparatus having a first 
operation unit for operating the operation panel and a second operation unit for 
operating a specific fimction of the controlled apparatus, and the control apparatus 
is adapted to receive a control signal from the operation apparatus, and determine; 
10 according to the control signal, whether or not to transmit a command for operating 
the specific ftinction from the control apparatus to the controlled apparatus . 

According to another aspect, the present invention which achieves this 
object r e lated to may be a communication control apparatus , comprising : ope r ation 
pan e l managing m e ans for managing an operation panel having a plui ' ality of 
display elements coi T csponding to p r cdctcmaincd o p e r ations and fo r sup p lying the 
ope r ation panel to a controller for accepting an operation for the operation panel, 
r e ceiving means for receiving a first control signal indicating that th e display 
element has b ee n op er at e d and a s e cond control signal for specifying any of 
sp e cific ope r ations f r om th e cont r olle r , and cont r ol means for controlling an 
operation corresponding to the display element to be executed on the basis of the 
first conti ' ol signal and controlling tlie sp e cific operation to be executed on the 
basis of the s e cond cont r ol signal that receives an operation panel for operating a 
controlled apparatus from the controlled apparatus, and displays the operation 
panel on a display unit, the control apparatus comprising: a receiving imit adapted 
25 to receive a control signal from an operation apparatus having a first operation unit 
for operating the operation panel and a second operation unit for operating a 
specific o p e r ation s fianction of the controlled apparatus, and a control unit adapted 
to determine, according to the control signal, whether or not to transmit a 
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command for operating the specific function from the control apparatus to the 
controlled apparatus . 

According to still another aspect, the present invention which achieves this 
object may be relat e d to a coainiunication method , comprising th e st e ps of : 
5 supplying an operation panel having a p lurality of display elements con - esponding 
to predetermined operations to a conti ' ollcr for accepting an operation for the 
ope r ation pan e l, and receiving a fi r st cont r ol signal indicating tliat any of the 
display e lements has b e en op er at e d o r a second cont r ol signal fo r specifying any of 
sp e cific operations, whe r ein a cont r ol is made so that an operation co rr esponding to 

10 th e display element is executed if the fi r st cont r ol signal is r eceived and the 

specific operation is executed if the second cont r ol signal is r eceived performed in 
a control apparatus that receives an operation panel for operating a controlled 
apparatus from the controlled apparatus, and displays the operation panel on a 
dis play unit, the method comprising the steps of receiving a control signal from an 

15 o peration apparatus having a first operation unit for operating the operation panel, 
and a second operation unit for operating a specific function of the controlled 
apparatus, and determining, according to the control signal, whether or not to 
transmit a command for operating the specific function from the control apparatus 
to the controlled apparatus . 

20 Other aspects of the present invention are set forth in this application, and 

are not disclaimed, whether or not thev may be claimed in the present application. 

Acco r ding to another aspect, the p r esent invention wliich achieves this 

object related to a communication apparatus, comprising : r eceiving means for 
re c e iving a signal f r om an op e ration appai - atus having a fi r st ope r ation unit fo r 

25 op e rating an op e ration panel having a plui ' ality of display el e ments corr e sponding 
to pr ed e termined operations su p plied f r om a conti ' olled d e vic e and a second 
o p e r ation unit for specifying specific operations, and transmitting means for 
tremsmitting a first control signal indicating that any of tlie display e l e ments has 
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b ee n operated on the basis of aii ope r ation fo r the first op e ration uiiit to the 
cont r olled device and fo r transmitting a second cont r ol signal for specifying any of 
tlic s p ecific op e rations on the basis of an operation for th e s e cond operation unit to 
tlic conti ' oll e d device. 

5 Acco r ding to still anothe r aspect, the present invention which achieves this 

obj e ct r elated to a communication appai - atus, comprising: first operation means for 
operating an ope r ation pan e l having a plurality of display elements co rr esponding 
to predetermined op er ations supplied f r om a conti - oUed device to a cont r oll er , 
second op e ration means fo r sp e cifying any of sp e cific operations, and transmitting 
10 means for t r ansmitting a signal indicating that the first ope r ation means has been 

operated and a signal indicating tliat the second operating means has be e n op e rated 
to the cont r olle r . 

Still other objects of the present invention, and the advantages thereof, will 
become fully apparent from the following detailed description of the embodiments. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of assistance in explaining an example of a remote 
control system according to [[an]] a preferred embodiment of the present 
invention; 

2 0 Fig. 2 is a diagram showing an example of an operation panel according to 

the a preferred embodiment of the present invention; 

Fig. 3 is a diagram showing an example of a remote control according to 
the a preferred embodiment of the present invention; 

Fig. 4 is a diagram of assistance in explaining an example of a data 
2 5 structure of the operation panel according to the a preferred embodiment of the 
present invention; 
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Fig. 5 is a diagram of assistance in explaining an example of a data 
structure of a panel element according to tiic a preferred embodiment of the present 
invention; 

Fig. 6 is a diagram of assistance in explaining an example of a protocol for 
5 a remote control system according to [[an]] a preferred embodiment of the present 
invention; 

Figs. 7A and 7B are diagrams of assistance in explaining an example of a 
frame structure of a control command and a response; 

Figs. 8A and 8B are diagrams of assistance in explaining a command frame 
10 of a GUI UPDATE command according to titc a preferred embodiment of the 
present invention; 

Figs. 9A, 9B and 9C are diagrams of assistance in explaining a command 
frame of a PUSH_GUI_DATA command according to tiic a preferred embodiment 
of the present invention; 
15 Fig. 10 is a diagram of assistance in explaining a command frame of a 

USER_ACTION command according to tiic a preferred embodiment of the present 
invention; 

Fig. 1 1 is a diagram of assistance in explaining a command frame of a 
DIRECT_ACTION command according to the a preferred embodiment of the 
2 0 present invention; 

Fig. 12 is a diagram showing an example of action_code; 

Fig. 1 3 is a diagram of assistance in explaining a processing procedure of a 
controlled device according to [[an]] a preferred embodiment of the present 
invention; and 

2 5 Fig. 14 is a diagram of assistance in explaining a processing procedure of a 

controller according to lite a preferred embodiment of the present invention. 
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The preferred embodiments of the present invention will now be described 
in detail hereinafter with reference to the accompanying drawings. 

Referring to Fig. 1, there is shown a block diagram of assistance in 
explaining an example of a remote control system according to this embodiment. 
5 In this system, reference numeral 1 designates a digital communication 

network. This network 1 is realized by a high-speed serial bus conforming to the 
IEEE1394-1995 standards and its extended standards (hereinafter, referred to as 
IEEE1394 standards). 

Reference numeral 100 designates a camera-integrated digital video 
1 0 recorder (hereinafter, a DVC) which is an example of a controlled device in this 
system. The DVC 1 00 has an operation panel for providing an operating 
environment characteristic of the DVC 100. 

Reference numeral 110 designates a tuner that is an example of a controlled 
device in this system. The tuner 110 has an operation panel for providing an 
1 5 operating environment characteristic of the tuner 1 1 0 in the same maimer as the 
DVC 100. 

The operation panel (also referred to as an operation screen) comprises an 
image or a text and it is a graphical user interface providing a user with a graphical 
operating environment easy to imderstand visually. 

2 0 Reference numeral 120 designates a digital television set (hereinafter, a 

digital television) which is an example of a controller in this system. The digital 
television 120 displays an operation panel of each controlled device to provide a 
graphical operating environment. 

Reference numeral 130 designates a remote controller (hereinafter, a remote 

2 5 control) for operating the operation panel displayed on the digital television 120. 
The remote control 130 converts an operation input of a user to a predetermined 
control signal and wirelessly transmits the control signal to the digital television 
120. The user can operate the operation panel displayed on the digital television 
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120 by using the remote control 130 to remotely operate a controlled device 
corresponding to the operation panel. 

First, a configuration of the DVC 100 will be described below. There are 
shown a digital interface (hereinafter, a 1394 interface) conforming to the 
5 IEEE 13 94 standards, a control unit 102, a Panel Management Department 103, and 
a DVC unit 104. 

The Panel Management Department 103 stores GUI (graphical user 
interface) data forming an operation panel of the DVC 100 in accordance with a 
data structure described later. If an operation intended for the operation panel is 

10 detected, the Panel Management Department 103 reads out (or generates) GUI data 
corresponding to the operation and supplies it to the 1394 interface 101. 

The DVC unit 104 records stream data (including digital video data, digital 
audio data, and text data) conforming to a predetermined data format on a storage 
medium 105 or reproduces desired stream data required by a user from the storage 

15 medium 105. The storage medium 105 comprises a magnetic tape, a magnetic 
disk, an optical disk, a semiconductor memory, and a hard disk or their 
combinations and is removable. As a predetermined data format, the DV mode, 
the MPEG2 mode, or other modes are used. The stream data recorded on the 
storage medium 105 by the DVC unit 104 is generated on the basis of an 

20 isochronous packet received by the 1394 interface 101 and the stream data 

reproduced from the storage medium 1 05 by the DVC unit 1 04 is supplied to the 
1394 interface 101. 

The 1394 interface 101 is connected to a 1394 interface of another device 
via the network 1. The 1394 interface 101 generates the isochronous packet on the 

2 5 basis of the stream data supplied by the DVC unit 104 and isochronously transfers 
this isochronous packet. In addition the 1394 interface 101 receives an 
asynchronous packet including a control command for controlling operations of the 
Panel Management Department 103, the DVC unit 104 and the like and supplies 
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the control command to the control unit 102. Furthermore, the 1394 interface 101 
generates an asynchronous packet on the basis of the GUI data supplied by the 
Panel Management Department 103 and asynchronously transfers this 
asynchronous packet. 

5 The control unit 102 has commimications via the network 1 to control the 

operations of the DVC unit 104 and the Panel Management Department 103. The 
control unit 1 02 reads out a program for realizing a processing procedure of this 
' embodiment from the internal storage medium and integrally controls respective 
units of the DVC 100 on the basis of the program. 

1 0 Next, a configuration of the tuner 110 will be described below. There are 

shown a 1394 interface 1 1 1, a control unit 1 12, a Panel Management Department 
1 13, and a tuner unit 1 14. 

The Panel Management Department 113 stores GUI data forming an 
operation panel of the tuner 110 and stores it in accordance with a data structure 

1 5 described later. If an operation intended for the operation panel is detected, the 
Panel Management Department 113 reads out (or generates) GUI data 
corresponding to the operation and supplies it to the 1394 interface 111. 

The tuner unit 114 selectively receives a desired digital broadcasting wave 
from a communication line network 115, selects desired stream data (including 

2 0 digital video data, digital audio data, and text data) from the received digital 

broadcasting wave, and supplies the selected stream data to the 1394 interface 111. 
The communication line network 115 comprises a ground-based broadcasting 
network, a satellite broadcasting network, a cable television broadcasting network, 
a public line network, and a mobile communication network or their combinations. 

2 5 The 1394 interface 1 1 1 is connected to a 1394 interface of another device 

via the network 1 . The 1394 interface 111 generates an isochronous packet on the 
basis of the stream data supplied by the tuner unit 114 and isochronously transfers 
the isochronous packet. In addition the 1394 interface 1 1 1 receives an 
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asynchronous packet including a control command for controlling operations of the 
Panel Management Department 113, the tuner unit 1 14, and the like and supplies 
this control command to the control unit 112. Furthermore, the 1394 interface 111 
generates an asynchronous packet on the basis of the GUI data supplied by the 
5 Panel Management Department 113 and asynchronously transfers this 
asynchronous packet. 

The control unit 112 has commxmications via the network 1 to control the 
operations of the tuner unit 1 14 and the Panel Management Department 113. The 
control unit 112 reads out a program for realizing a processing procedure of this 

1 0 embodiment from the internal storage medium and integrally controls respective 
units of the tuner imit 1 14 on the basis of the program. 

Next, a configuration of the digital television 120 will be described below. 
There are shown a 1394 interface 121, a control unit 122, a stream data-processing 
unit 123, a Panel Management Department 124, a drawing control imit 125, a 

15 remote control control unit 126, an image composition unit 127, a display unit 128, 
a voice output unit comprising a speaker 129. 

The stream data-processing unit 123 generates stream data from the 
isochronous packet received by the 1394 interface 121 and then separates this 
stream data into video stream data and audio stream data. The stream data- 

2 0 processing xmit 123 further decodes each stream data to supply the video data to the 
image composition unit 127 and to supply the audio data to the voice output unit 
129. 

The voice output unit 129 digital-analog-converts the audio data supplied 
by the stream data-processing unit 123 and amplifies it to be output. 
2 5 The Panel Management Department 124 stores the operation panels which 

have been transferred from respective controlled devices and manages them for 
each controlled device. 
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The drawing control unit 125 controls drawings of the respective operation 
panels. Specifically it controls a display position, a display size, a display time, 
rewriting of displayed contents, a focal navigation and the like of each operation 
panel. 

5 The image composition unit 127 synthesizes the video data supplied by the 

stream data-processing unit 123 with the operation panel supplied by the drawing 
control unit 125 and digital-analog-converts the composed video data so as to be 
supplied to the display unit 128. 

The display unit 128 comprises a predetermined display and displays a 

10 video signal supplied by the image composition unit 127 on this display. The 

predetermined display is a CRT (cathode-ray tube), a liquid crystal panel, a plasma 
display panel, a projection-type display panel, or the like. 

The remote control control unit 1 26 communicates with the remote control 
130 via a transmission medium such as an infrared ray or a radio wave. The 

1 5 remote control unit 126 receives a control signal from the remote control 130 and 
supplies this control signal to the control unit 122. 

The 1394 interface 121 is connected to a 1394 interface of another device 
via the network 1. The 1394 interface 121 receives an asynchronous packet 
including GUI data and supplies this GUI data to the Panel Management 

20 Department 124 and the drawing control imit 125. The 1394 interface 121 

asynchronously transfers a control command obtained by operating the remote 
control 130 to a controlled device. Furthermore, the 1394 interface 121 receives an 
asynchronous packet including a control command for controlling operations of the 
stream data-processing unit 123, the Panel Management Department 124, the 

2 5 drawing control unit 125 and the like and supplies this control command to the 
control unit 122. 
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The control unit 122 reads out a program for realizing a processing 
procedure of this embodiment from the internal storage medium and integrally 
controls respective vmits of the digital television 120 on the basis of the program. 

Basic functions of the 1394 interfaces 101, 102, and 103 will be described 

5 below. 

These 1394 interfaces are capable of coping with a daisy-chain connection, 
a node branch connection, and their combinations, thus enabling connections 
having a high degree of freedom. Respective transfer rates are 100 Mbps, 200 
Mbps, and 400 Mbps and data is transferred in a serial transfer mode. An interface 
1 0 having a high-order transfer rate is free to be connected to interfaces having their 
maximum transfer rates different from each other so as to support low-order data 
transfer rates. 

In addition, the 1394 interfaces support two transfer modes; an 
asynchronous transfer mode and an isochronous transfer mode. The asynchronous 

1 5 transfer mode is appropriate for transferring a control command, file data or other 
data. The isochronous transfer mode is used for ensuring a certain amount of a 
data transfer per predetermined transfer cycle (a single transfer cycle is approx. 125 
[is). The isochronous transfer mode is appropriate for transferring data expected to 
have a temporal continuity (what is called stream data) such as video data and 

2 0 audio data. In a single transfer cycle period, the isochronous transfer mode has a 
higher priority than the asynchronous transfer mode. 
(Description of operation panel) 

Referring to Fig. 2, there is shown a diagram of a sample operation panel 
according to this embodiment. In Fig. 2, an example is given for a description of 
2 5 displaying an operation panel of the DVC 100 which is one of the controlled 
devices on the display unit 128 of the digital television 120, for example. 
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In Fig. 2, there are shown a display screen 200 for the display unit 128, 
stream data 201 isochronously transferred by the DVC 100, and an operation panel 
as a panel element 202 for the DVC 100. 

Reference numerals 203 to 207 designate button elements. The button 
5 element 203, the button element 204, the button element 205, the button element 
206, and the button element 207 correspond to a rewind function, a reproduction 
function, a fast-forward function, a stop function, and an image recording function, 
respectively. Reference numeral 208 designates an animation element. 

Reference numerals 209 to 212 designate icon elements. The icon elements 

1 0 209 to 212 correspond to respective contents recorded on a storage medium to be 
reproduced by the DVC 100. If the remote control 130 selects one of these icon 
elements 209 to 212, the DVC 100 starts a reproduction of a content corresponding 
to the icon element at 209 to 212 and the reproduced content is isochronously 
transferred to the DVC 100. 

15 Reference numeral 213 designates a focus. The focus 213 gives visual 

expressions to the elements 203 to 212 by means of a highlighting display or a 
color change. A position of the focus 213 moves in response to an operation of the 
remote control 130. Processing of operating the focus in response to an operation 
of the remote control 130 is referred to as focal navigation. 

2 0 While the embodiment has been described by giving an example of 

displaying the operation panel of the DVC 100 on the display unit 128 of the 
digital television 120 in Fig. 2, it is also possible to display an operation panel of 
the tuner 110. 

(Data structure of operation panel) 

2 5 Referring to Fig. 4 and Fig. 5, there are shown examples of data structures 

of operation panels according to this embodiment. Fig. 4 shows a diagram of a 
data structure of the operation panel 202. As shown in Fig. 4, GUI elements 202 to 
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212 forming the operation panel 202 have a tree structure routed from the panel 
element 202. Fig. 5 shows a diagram of a data structure of the panel element 202. 

In Fig. 4, the panel element 202 retains a link with text data showing a title 
and a functional description, a display position, a display size, and links with the 
5 elements 203 to 212. Each of the button elements 203 to 207 retains a link with 
text data showing a title and a functional description, a display position, a display 
size, and links with bit map image data corresponding to "depression" and 
"release" user actions. 

In addition, in Fig. 4, the animation element 208 retains a link with text 
1 0 data showing a title and a functional description, a display position, a display size, 
and four links with bit map image data. Each of the icon elements 209 to 212 
retains a link with text data showing a title and a functional description, a display 
position, a display size, and bit map image data corresponding to a "selection" user 
action. 

15 As shown in Fig. 4 and Fig. 5, an element ID is given to each GUI element 

and actual data linked therewith. 

While the data structures of the operation panels of the DVC 100 have been 
described in Fig. 4 and Fig. 5, operation panels of the tuner 110 have the same data 
structures as shown in Fig. 4 and Fig. 5. 
2 0 (Description of remote control) 

Referring to Fig. 3, there is shown a diagram of an example of a remote 
control 130 according to this embodiment. 

In Fig. 3, there are shown operation keys 301 to 305 for specifying 
movements of the focus 213. The reference numeral 301 designates a direction key 
2 5 for specifying a movement in an upper direction. The focus 213 displayed on the 
operation panel moves upward according to an operation of the key 301 . The 
reference numeral 302 designates a direction key for specifying a movement to the 
left. The focus 213 displayed on the operation panel moves to the left according to 
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an operation of the key 302. The reference numeral 303 designates a direction key 
for specifying a movement in a lower direction. The focus 213 displayed on the 
operation panel moves downward according to an operation of the key 303. The 
reference numeral 304 designates a direction key for specifying a movement to the 
5 right. The focus 213 displayed on the operation panel moves to the right according 
to an operation of the key 304. The reference numeral 305 designates a 
determination key for specifying a selection or a decision. 

Control signals corresponding to these keys 301 to 305 are supplied to the 
control unit 122 via the remote control control unit 126 of the digital television 
10 120. The control imit 122 controls movements of the focus 213 on the basis of the 
control signals corresponding to the keys 301 to 304. 

The control unit 1 22 detects one of the "depression", "release", and 
"selection" user actions on the basis of a control signal corresponding to the key 
305. For example, if the focus 213 is adjusted to one of the button elements, the 
15 "depression" or "release" user action is detected. If the focus 213 is adjusted to 
one of the icon elements or the animation element, the "selection" user action is 
detected. The detected user action is converted to a USER_ACTION command 
described later and then asynchronously transferred to a controlled device. 

In the above, the "depression" means that the button element to which the 
2 0 focus 213 is adjusted has been put in a selected condition, the "release" means that 
the button element to which the focus 213 is adjusted to has been put in a non- 
selected condition, and "selection" means that the icon element or the animation 
element to which the focus 213 is adjusted has been put in a selected condition. 

A method of operating the operation panel by using the keys 301 to 305 for 
2 5 remotely operating the controlled device as set forth in the above is referred to as a 
focal navigation mode in this embodiment. 

Furthermore, in Fig. 3, there are shown operation keys 31 1 to 315 
associated with operations corresponding to specific functions, where operations 
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corresponding to the specific functions covered by the keys 31 1 to 315 depend 
upon an operation panel currently under the operation. 

The reference numeral 311 designates an operation key for specifying a 
movement in a past, lower, or backward direction or a start or the like of 
5 rewinding. The reference numeral 312 designates an operation key for specifying a 
start or the like of reproduction. The reference numeral 313 designates an 
operation key for specifying a movement in a future, upper, or forward direction or 
a start or the like of a fast-forward operation. The reference numeral 314 
designates an operation key for specifying a stop or the like of reproduction, 

1 0 rewinding, fast-forward or other operations. The reference numeral 315 designates 
an operation key for specifying a start or the like of recording. 

Control signals corresponding to these keys 31 1 to 315 are supplied to the 
control unit 122 via the remote control control unit 126 of the DTV 110. The 
control unit 122 determines a currently selected operation panel and detects which 

1 5 key among the keys 3 1 1 to 3 1 5 has been depressed on the basis of the control 
signal before generating a DIRECT_ACTION command described later. This 
DIRECT_ACTION command is asynchronously transferred to the controlled 
device. 

A method of remotely operating a specific function of the controlled device 
2 0 with a single-action button operation by using the keys 301 to 305 without any 
operation of the operation panel is referred to as a direct action mode in this 
embodiment. The direct action mode requires less operation procedures in 
comparison with the focal navigation mode and enables the controlled device to be 
remotely operated even if the operation painel is not displayed. As set forth 
2 5 hereinabove, there are provided two operation methods, the focal navigation mode 
and the direct action mode in this embodiment. 

Furthermore, in Fig. 3, there are shown operations keys 321 to 324 for 
specifying operations on the DTV 110. The reference numeral 321 designates an 
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operation key for specifying the power supply of the DTV 1 10 to be turned on or 
off. The reference numeral 322 designates an operation key for specifying an 
increase of the volume of a sound or voice outputted by the DTV 110. The 
reference numeral 323 designates an operation key for specifying a decrease of the 
5 volume of a sound or voice outputted by the DTV 110. The reference numeral 324 
designates an operation key for calling a menu screen for adjusting various 
functions of the DTV 110. On the menu screen, there are items for selecting a 
display or a non-display of an operation panel and items for selecting a desired 
operation panel. A user can change the display to the non-display or vice versa of 
1 0 the operation panel or can select an operation panel to be operated by operating 
these items. 

Control signals corresponding to these keys 321 to 324 are supplied to the 
control imit 122 via the remote control control unit 126 of the DTV 110. The 
control unit 122 controls the power supply to be tumed on or off, a volume 
1 5 adjustment, a menu screen call, or the like on the basis of the control signals. 

As set forth hereinabove, the remote control 130 are provided with the first 
operation keys 301 to 305 for operating the operation panel of the controlled device 
and the second operation keys 3 1 1 to 3 1 5 directly associated with the specific 
functions of the controlled device, thereby enabling two types of operation methods 
20 to be provided. 

(Description of protocol) 

Referring to Fig. 6, there is shown a diagram of an example of a protocol of 
a remote control system according to this embodiment. By referring to Fig. 6, there 
is provided a description of a protocol between the digital television 120 which is a 
2 5 controller and the DVC 100 which is one of the controlled devices. 

The 1394 interface 121 of the digital television 120 initially establishes an 
isochronous connection between the digital television 120 and the DVC 100 in 
accordance with a predetermined connection establishment procedure (601 in Fig. 
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6). The DVC 100 isochronously transfers the stream data to the digital television 
120 via this isochronous connection. 

Specifically, after accessing an IRM (isochronous resource manager; the 
digital television 120 realizes this function here, for example) of a 1394 network 
5 and securing a channel and a band for the isochronous transfer, a peer-to-peer 
connection (a one-for-one logical connection) is established between an 
isochronous plug (w^hich functions as a virtual output plug) of the 1394 interface 
101 of the DVC 100 and an isochronous plug (which functions as a virtual input 
plug) of the 1394 interface 121 of the digital television 120. 

10 Next, the 1394 interface 121 of the digital television 120 establishes an 

asynchronous connection between the digital television 120 and the DVC 100 in 
accordance with a predetermined connection establishment procedure (602 in Fig. 
6). The DVC 100 asynchronously transfers GUI data of an operation panel to the 
digital television 120 via this asynchronous connection. 

15 The 1394 interface 121 of the digital television 120 issues a GUI_UPDATE 

(start) command (603 in Fig. 6). This GUI_UPDATE (start) command is used for 
specifying a start of transferring the operation panel of the DVC 100. 

The 1394 interface 121 of the digital television 120 issues a 
PUSH_GUI_DATA command (604 in Fig. 6). This PUSH_GUI_DATA command 

2 0 is used for requesting the operation panel of the DVC 100. 

After receiving the PUSH_GUI_DATA command, the DVC 100 reads out 
the GUI data of its own operation panel from the Panel Management Department 
103 and supplies it to the 1394 interface 101. The 1394 interface 101 sequentially 
transfers asynchronously this GUI data to the digital television 120 via the 

25 asynchronous connection (605 in Fig. 6). 

After receiving the GUI data of the DVC 100, the digital television 120 
displays the operation panel of the DVC 100 on the basis of the GUI data and 
provides a user with a graphical operating environment (606 in Fig. 6). 
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Specifically, the digital television 120 stores the GUI data of the DVC 100 
in the Panel Management Department 124. The control unit 122 reads out this GUI 
data for each GUI element and supplies it to the drawing control unit 125. The 
drawing control unit 125 analyzes a data structure of each GUI element, checks a 
5 position, a size, and contents for a display, and then controls the image 

composition unit 127 on the basis of the analysis result. The image composition 
unit 127 synthesizes the GUI element supplied by the drawing control unit 125 
with video data supplied by the stream data-processing unit 123 and digital-analog- 
converts the composed video data to a predetermined video signal. The display 

1 0 unit 128 provides a user with a graphical operating environment as shown in Fig. 2, 
for example, by displaying this video signal. 

After displaying the operation panel of the DVC 1 00, the digital television 
120 enables the above two operation methods (namely, the focal navigation mode 
and the direct action mode) to be executed. 

15 In the focal navigation mode, a focal navigation is performed in accordance 

with a user operation and a user action ("depression", "selection", etc.) permitted 
for each GUI element is accepted. On the other hand, in the direct action mode, a 
direct action is accepted in accordance with any of the keys 3 1 1 to 315 of the 
remote control 130 independently of whether there is a display of the operation 

20 panel. 

If the user action is detected, the 1394 interface 121 issues a 
USER_ACTION command (607 in Fig. 6). This USER_ACTION command is 
used for indicating what user action has occurred for a certain GUI element. There 
will be described hereinafter a case of issuing the USER_ACTION command 
2 5 indicating that a "depression" user action has occurred to the icon element 209, for 
example. 

After receiving the USER_ACTION command, the DVC 100 controls the 
DVC unit 104 on the basis of a GUI element and a user action specified by the 
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USER_ACTION command (608 in Fig. 6). For example, the DVC unit 104 
retrieves and reproduces contents corresponding to the icon element 209 from the 
storage medium 105 and supplies these contents as stream data to the 1394 
interface 101. 

5 If the GUI of the operation panel changes in accordance with the 

USER_ACTION command, the DVC 100 reads out the GUI data corresponding to 
the change portion from the Panel Management Department 103 and transfers the 
GUI data to the digital television 120 via the asynchronous connection (609 in Fig. 
6). 

1 0 The DVC 100 transfers the contents corresponding to the icon element 209 

via the isochronous connection (610 in Fig. 6). 

After receiving the GUI data and the contents corresponding to the icon 
element 209, the digital television 120 updates the operation panel of the DVC 100 
on the basis of the received GUI data and synthesizes the updated operation panel 

15 with the received contents before displaying (611 in Fig. 6). 

On the other hand, if the direct action is detected, the 1394 interface 121 
issues a DIRECT_ACTION command (612 in Fig. 6). 

Receiving the DIRECT_ACTION command, the DVC 100 controls 
specific functions specified by the DIRECT_ACTION command (613 in Fig. 6). 

2 0 In the above, if the specific function specified by the DIRECT_ACTION 

command is invalid for the DVC 100 which is a controlled device, in other words, 
if it is not included in services provided by the DVC 100 or cannot be temporarily 
executed due to an internal condition, an error response indicating a non- 
executable condition is returned. 

25 If the GUI of the operation panel changes in accordance with the 

DIRECT_ACTION command, the DVC 100 reads out the GUI data corresponding 
to the change portion from the Panel Management Department 1 03 and transfers 
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the GUI data to the digital television 120 via the asynchronous connection (614 in 
Fig. 6). 

The digital television 120 updates the operation panel of the DVC 100 on 
the basis of the received GUI data (615 in Fig. 6). In the above, if the operation 
5 panel is in a non-display condition, the digital television 120 displays the updated 
operation panel on the display screen until the user specifies a non-display 
condition or until a predetermined period of time elapses. This configuration 
enables the result of the operation to be graphically expressed on the operation 
panel even if an operation is executed in the direct action mode. In addition, a user 

10 can visually confirm which function of the DVC 100 has been actually operated, by 
which the current operating condition can always be grasped. 

The digital television 120 and the DVC 100 repeatedly execute the 
procedure in 604 to 615 to accept the operations in the focal navigation mode and 
the direct action mode. This enables the user to use two operation methods 

1 5 together, by which a plurality of operations can be continuously performed in a 
short time. Therefore, for example, it is effective to repeat the reproduction, fast- 
forward, reproduction, and fast-forward operations in a short time. In addition, for 
example, it is also effective to continuously operate GUI elements located apart 
from each other on the operation panel in a short time. 

20 If the operation in the focal navigation mode and the direct action mode is 

terminated, the 1394 interface 121 of the digital television 120 issues a 
GUI_UPDATE (stop) command (616 in Fig. 6). 

Afterward, the asynchronous connection is disconnected and then the 
isochronous connection is disconnected (617 and 618 in Fig. 6). 

2 5 While there has been described the protocol between the digital television 

120 and the DVC 100 in Fig. 6, the present invention is not limited to this. The 
tuner 1 10 that is another controlled device can communicate with the digital 
television 120 in the same manner as for this protocol. Furthermore, a 
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communication between the digital television 120 and the DVC 100 can be 
executed in parallel with a communication between the digital television 120 and 
the tuner 110. 

Next, the above GUI_UPDATE command, the USER_ACTION command, 
5 and the DIRECT_ACTION command vAll be described below. The control 

commands are transferred on the basis of the function control protocol stipulated in 
the IEC61 883-1. 

Fig. 7A shows a control command stipulated by the function control 
protocol and Fig, 7B shows a frame structure of a response corresponding thereto. 

1 0 "ctype" and "response" indicate a command type and a response type, respectively. 
The "subunit_type" and "subunit_ID" are used for specifying a sub-unit to be 
controlled, "opcode" indicates controlled contents and operand[l] to operand[n] 
are defined according to the contents of opcode. 

(Description of GUI_UPDATE command) 

1 5 Referring to Fig. 8 A, there is shown a diagram of assistance in explaining a 

command frame of the GUI_UPDATE command according to this embodiment, 
"source plug" is stored in operand[l] and "subfunction" indicating an instruction 
for the controlled device is stored in operand[3]. 

The content of "subfunction" is described below by using Fig. 8B. An 

2 0 "open" code is used for giving an instruction on a start of a remote operation with 
an operation panel (hereinafter, simply referred to as a remote operation). A 
"close" code is used for giving an instruction on an end of the remote operation. 

A "start" code is used for giving an instruction on a start of transferring 
GUI data forming the operation panel. A "stop" code is used for giving an 

2 5 instruction on a stop of transferring the GUI data. The controlled device transmits 
GUI data depending upon changes in a device condition or in response to a result 
of an operation until this GUI UPDATE (stop) command is received. 
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A "restore" code is used for giving an instruction on a return of an 
asynchronous connection after a bus reset occurs in the network 1 . A "change" 
code is used for giving an instruction on a change of a display range of an 
operation panel. 
5 (Description of PUSH_GUI_DATA command) 

Referring to Fig. 9A, there is shown a diagram of assistance in explaining a 
command frame of the PUSH_GUI_DATA command according to this 
embodiment. A plug number obtained by the GUI_UPDATE (open) command is 
stored in operand[0]. An "indicator" is stored in operand[5]. Element IDs for 
1 0 identifying GUI elements are stored in operand[6] to operand[9]. 

The contents of the "indicator" will be described below by using Fig. 9B. 
The "indicator" comprises "level" and "with_data". A code is set to "level" for 
specifying a range of GUI data requested of the controlled device by the controller. 
If actual data (a text, an image, a sound, etc.) of the GUI element specified by using 
1 5 the element ID and "level", "1" is stored in "with_data". 

The contents of "level" will be described by using Fig. 9C. An "itself 
code is used for requesting the GUI element itself specified by the element ID. An 
"itself and next level" code is used for requesting the GUI element specified by the 
element ID and a GUI element belonging to the next level of the GUI element and 
20 an "all level" code is used for requesting all the GUI elements. 
(Description of USER_ACTION command) 

Referring to Fig. 10, there is shown a diagram of assistance in explaining a 
command frame of a USER_ACTION command according to this embodiment. In 
operand[0], operand[3] to operand[6], operand[7], and operand[8] or after, 
2 5 "source_plug", "element__id", "action_type", and "data" are stored, respectively. 

The "element_id" code is used for indicating an element ID for identifying 
an GUI element. The "action_type" code is used for specifying a user action such 
as a depression, a release, and a selection. 
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(Description of DIRECT_ACTION command) 

Referring to Fig. 1 1, there is shown a diagram of assistance in explaining a 
command frame of a DIRECT ACTION command according to this embodiment. 
In operand[0], operand[2], and operand[3] or after, "action_code", "action type", 
5 "action_data_length", and "action_specified_data" are stored, respectively. 

The "action code" code is used for indicating a specific function. The 
"action_type" code is used for indicating how a specific function is operated. The 
"action_data_length" code is used for indicating a data length of 
"action_specified_data". The "action_specified_data" code is used for indicating 
1 0 data inherent in the "action_code" code. 

Referring to Fig. 12, there is shown a diagram of an example of the 
"action code" code shown in Fig. 1 1 . 

If the currently selected operation panel is an operation panel of the DVC 
100 in this embodiment, the "action_code" corresponding to the key 31 1 is "28h", 
1 5 which indicates rewinding of the storage medium 1 05, for example. The 

"action_code" corresponding to the key 312 is "24h", which indicates a start of 
reproduction of the storage medium 105. The "action_code" corresponding to the 
key 313 is "29h", which indicates fast-forwarding of the storage medium 105. The 
"action_code" corresponding to the key 314 is "25h", which indicates a stop of an 
2 0 operation of the storage medium 105. The "action_code" corresponding to the key 
315 is "27h", which indicates a start of recording into the storage medium 105. 

On the other hand, if the currently selected operation panel is an operation 
panel of the tuner 110, the "action_code" corresponding to the key 3 1 1 is "2Bh", 
which indicates a movement in a lower direction of a receiving channel. The 
2 5 "action_code" corresponding to the key 3 12 is "06h", which indicates a display of 
EPG (electric program guide). The "action_code" corresponding to the key 3 1 3 is 
"2Ch", which indicates a movement in an upper direction of the receiving channel. 

(Description of processing procedure of controlled device) 
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Referring to Fig. 13, there is shown a diagram of assistance in explaining a 
processing procedure of a controlled device according to this embodiment. While 
a processing procedure of the DVC 100 that is one of the controlled devices is 
described here, the tuner 110 can also be processed as described below. 
5 In step SI 01 the 1394 interface 101 receives a control command 

corresponding to a user remote control operation. This type of control commands 
include the above-described USER_ACTION command, the above-described 
DIRECT_ACTION command, and other control commands. 

Next, in step SI 02, the control unit 102 determined whether the received 
1 0 control command is the USER_ACTION command. If it is the USER_ACTION 
command, step SI 03 and subsequent processing is executed; otherwise, step SI 09 
and subsequent processing is executed. 

In step SI 03, the control unit 102 checks contents of the received 
USER ACTION command and determines whether it is an executable action. If it 
15 is executable, the control unit 102 executes processing of step SI 04; otherwise, the 
control unit 102 executes processing of step SI 08. In the step SI 08, an error 
response is generated for indicating that the received USER_ACTION command 
cannot be processed and the error response is returned to the controller. 

In step S104, the 1394 interface 101 normally receives the USER_ACTION 
2 0 command and returns a response indicating that the command is executable to the 
controller. 

Next, in step SI 05, the control unit 102 controls a function corresponding 
to an operation panel specified by the "element_id" code of the USER_ACTION 
command according to the "action_type" code of the USER_ACTION command. 
2 5 In step SI 06, the control unit 102 determines whether the operation panel 

operated by the controller should be changed together with an execution of the 
USER_ACTION conmiand. If it should be changed, processing of step SI 07 is 
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executed; otherwise, the processing control returns to the step SlOl to receive the 
next control command. 

In the step SI 07, the 1394 interface 101 transmits the GUI data of a portion 
to change to the controller. Afterward, the controlled device returns to the step 
5 S 101 to receive the next control command. 

On the other hand, the control unit 102 determines whether the received 
control command is a DIRECT_ACTION command in step SI 09. If it is the 
DIRECT_ACTION command, step SllO and subsequent processing is executed; 
otherwise, step SI 16 and subsequent processing is executed. In step SI 16, 
1 0 processing corresponding to the received control command is executed and then 
the processing control returns to the step SI 01 to receive the next control 
command. 

In the step SllO, the control unit 102 checks contents of the received 
DIRECT_ACTION command and determines whether it is an executable action. If 

15 it is executable, the control unit 102 executes processing of step Sill; otherwise, 
the control unit 102 executes processing of step SI 15. In the step SI 15, an error 
response is generated for indicating that the received DIRECT_ACTION command 
cannot be processed and the error response is returned to the controller. 
In step SI 12, the 1394 interface 101 normally receives the 

2 0 DIRECT ACTION command and returns a response indicating that the command 
is executable to the controller. 

Next, in the step SI 12, the control unit 102 controls a function directly 
specified by the "action_code" code of the DIRECT_ACTION command according 
to the "action_type" code of the DIRECT_ACTION command. 

25 In Step S 1 1 3, the control unit 1 02 determines whether the operation panel 

displayed on the controller should be changed together with an execution of the 
DIRECT_ACTION command. If it should be changed, processing of step SI 14 is 
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executed; otherwise, the processing control returns to the step SlOl to receive the 
next control command. 

In the step SI 14, the 1394 interface 101 transmits the GUI data of a portion 
to change to the controller. Afterward, the controlled device returns to the step 
5 S 1 01 to receive the next control command. 

In the above processing procedure, each controlled device can execute both 
of the USER_ACTION command and the DIRECT_ACTION command, and if an 
operation panel changes according to these control commands, GUI data of a 
portion corresponding to the change can be transmitted to the controller. 
1 0 (Description of processing procedure of controller) 

Referring to Fig. 14, there is shown a diagram of assistance in explaining a 
processing procedure of the digital television 120 that is a controller according to 
this embodiment. 

In step S201, the control unit 122 selects an operation panel to be operated 
1 5 according to a user remote control operation and displays the operation panel 

selected in accordance with the above-described protocol on the display unit 128. 

In step S202, the remote control control vmit 126 receives a control signal 
from the remote control 130. 

In step S203, the control imit 122 checks the received control signal and 
2 0 determines whether it is an operation of a focus. If it is an operation of the focus, 
the above-described focal navigation is executed in step S208. Otherwise, 
processing of step S204 is executed. 

In step S204, the control unit 122 checks the received control signal and 
determines whether it is an operation in the user action mode. If it is an operation 
25 in the focal navigation mode, a USER_ACTION command corresponding to the 
received control signal is generated and processing of step S205 is executed. 
Otherwise, processing of step S209 is executed. 
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In step S205, the 1394 interface 121 issues the USER_ACTION command 
to a controlled device corresponding to the cxorrently operated operation panel. 

In step S206, the 1394 interface 121 determines whether the GUI data of a 
portion to change on the operation panel has been received. If the GUI data of a 
5 portion to change is received, the operation panel is changed on the basis of the 

GUI data in the step S206. If the operation panel is in the non-display condition at 
this time, the changed operation panel is displayed on the display screen until a 
user specifies the non-display condition or until a predetermined period of time 
elapses. 

10 In step S209, the control unit 122 checks the received control signal and 

determines whether it is an operation in the direct action mode. If it is the 
operation in the direct action mode, the DIRECT ACTION command 
corresponding to the received control signal is generated and processing of step 
S210 is executed. Otherwise, a control of the digital television 120 or the like is 

1 5 executed in step S2 1 3 . 

In the step S210, the 1394 interface 121 issues a DIRECT_ACTION 
command to a controlled device corresponding to the currently operated operation 
panel. 

In step S21 1, the 1394 interface 121 determines whether GUI data of a 
2 0 portion to change on the operation panel has been received. If the GUI data of a 
portion to change is received, the operation panel is changed on the basis of the 
GUI data in step S212. If the operation panel is in the non-display condition at this 
time, the changed operation panel is displayed on the display screen until the user 
specifies the non-display condition or until a predetermined period of time elapses. 
25 In the above-described processing procedure, the controller can remotely 

operate a controlled device using two operation methods together. In addition, if 
either of the operation methods is used for an operation, a result of the operation 
can be graphically expressed on the operation panel. Furthermore, the user can 
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visually confirm which function of the controlled device has been actually 
operated, by which the current operating condition can always be grasped. 

As set forth hereinabove, according to this embodiment, two operation 
methods of the focal navigation mode and the direct action mode are used together, 
5 by which it becomes possible to flexibly cope with operating environments for a 
plurality of types of electronic devices and to remotely operate these electronic 
devices in a simple operation procedure. 

Furthermore, according to this embodiment, two operation methods of the 
focal navigation mode and the direct action mode are used together, by which a 
1 0 user can execute both of the first operation of selecting a desired fiinction while 

viewing an operation panel and the second operation of directly selecting a desired 
function without operating an operation panel. This reduces a complication of an 
operation, thereby significantly improving faciUty in operation. 

In addition, according to this embodiment, a plurality of operations can be 
1 5 continuously performed in a short time by using these two operation methods 
together. 

Furthermore, according to this embodiment, results of operations in the two 
operation methods are always reflected on an operation panel, by which a user 
easily check what condition the operation is in at present so as to prevent the user 
2 0 fi"om being confused. 

Still fiirther, according to this embodiment, a function corresponding to an 
operation key for realizing the direct action mode can be changed for each 
operation panel. This makes it possible to cope with various types of controlled 
devices flexibly without increasing the number of operation keys. 
2 5 The invention may be embodied in other specific forms without departing 

from essential characteristics thereof. 

For example, while the digital television 120 has been described as an 
example of a controller in this embodiment, the controller is not limited to the 
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digital television. Only if it is an electronic device having an operation apparatus 
equivalent to the remote control 130 and a display, any device can be used for it. 

In addition, while the DVC 100 and the tuner 110 have been described as 
examples of controlled devices in this embodiment, controlled devices are not 
limited to these devices. They can be either of audio-visual devices other than the 
DVC 100 and the tuner 110 (for example, a digital video recorder, a set-top box, 
etc.) and household electric appliances. 

Therefore, the above-mentioned embodiments are merely examples in all 
respects, and must not be construed to limit the invention. 

The scope of the present invention is defined by the scope of the appended 
claims, and is not limited at all by the specific descriptions of this specification. 
Furthermore, all the modifications and changes belonging to equivalents of the 
claims are considered to fall within the scope of the present invention. 
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